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ABSTRACT 

This report documents an experiment to determine the 
training effectiveness of a microcomputer-based avionics system 
trainer as a cost-effective alternative to training in the actual 
aircraft. Participants — 26 operationally qualified C-141 pilots with 
no prior knowledge of the Fuel Saving Advisory System (FSAS), a 
computerized fuel management system for C-141 and C-5A aircraft — were 
randomly assigned either to an experimental group to receive training 
on the FSAS via the microcomputer-based, self-tutorial trainer or to 
a control group to be trained by an instructor on the flight deck of 
an FSAS-eguipped C-141 aircraft. Both groups were trained to 
criterion proficiency on eight basic FSAS tasks, and time required to 
achieve proficiency was measured for all subjects on all tasks. As a 
result of training, the experimental group achieved a mean 
proficiency rating across the eight tasks which was significantly 
superior to that of the control group. Moreover, a transfer 
effectiveness ratio applied to assess transfer of training from the 
experimental trainer to the criterion task in the aircraft indicated 
the experimental trainer was fully as effective as aircraft training 
for the tasks and yielded a cost benefit ratio of 4.76 to 1 (aircraft 
training cost versus microcomputer trainer cost), which demonstrates 
the potential benefit of the experimental trainer to reduce training 
cost. Additional data are presented in two tables, and references are 
provided. The trainer tryout questionnaire and cost data 
documentation are appended. (Author/KM) 
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This report documents the evaluation of a low-cost trainer as a substitute for aircraft-based 
training. The training was provided to C-141 pilots to train them to perform several procedural 
tasks associated with the operation of the Fuel Savings Advisory System (FSAS). This system 1s 
being retrofitted on C-141 and C-5A aircraft. Operational aircrews must be trained to operate 
the FSAS 1n order to be mission qualified. This report documents an experiment 1n which trainees 
learned to operate the FSAS, either 1n the aircraft or via the low-cost alternative. Results 
showed both groups required the same amount of training time to learn to perform the tasks. 
Thus, the low-cost alternative was shown to be an effective substitute for the aircraft for 
training the FSAS tasks. A cost analysis comparing the two training methods shows the low-cost 
trainer 1s capable of avoiding more than 8 million dollars 1n training costs If 1t 1s enployed to 
train current C-141 and C-5A aircrews in the FSAS tasks. 
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PREFACE 



This effort represents a portion of the research and development (R&D) program of 
the A1r Force Human Resources Laboratory (AFHRL) for Technical Planning Objective 3, the 
thrust of which Is Aircrew Training* The general objective of this thrust 1s to 
Identify and demonstrate cost effectiveness In training A1r Force aircrew members. More 
specifically, the effort was part of the R&D conducted under the Aircrew Training 
Effectiveness subthrust, which has as Its goal the provision of a technology base for 
Improving the effectiveness and efficiency of training combat aircrews. The present 
effort was conducted as a part of Work Unit 1123-25-01, Special Function Trainer 
Technology. The research was accomplished 1n cooperation with the Military Airlift 
Command (MAC) In accordance with terms of a Memorandum of Agreement (MOA) between HQ 
MAC/DOT and AFHRL. Under the agreement, MAC will produce one or more experimental 
special -function trainers and AFHRL will provide consultative services to MAC for the 
design, development, and evaluation of the trainers. The purpose of the present effort 
was to evaluate the training effectiveness of one such special function trainer 
developed by MAC to teach avionics procedural skills to aircrew members. 

The author acknowledges the design and development efforts of MAC personnel which 
underlie the present report. Specifically acknowledged Is the work of: Lt Col Joe 
Burch, HQ MAC/DOT, who provided the Impetus for MAC's Investigation of special function 
trainer technology; Capt John Queern, HQ MAC/D00AE, who developed TEACHER, an 
Instructional authoring language that made possible the development of computerized 
training sequences by a MAC subject-matter expert without the need for a professional 
computer programmer; Capt Frank Murray, HQ MAC/DOT, who developed G-TEACHER, making 
possible the simulation of the avionics system required for the present effort, and who 
also provided the extensive documentation of training cost data contained In Appendix B 
of this report; and Maj Jacob Miller, 63 MAW/DOT, who developed the Fuel Savings 
Advisory System (FSAS) training syllabus, developed and validated the experimental 
software for the special function trainer, and provided the field training data 1n 
accordance with the requirements for the present analysis and evaluation. 
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LOW-COST AVIONICS SIMULATION FOR AIRCREW TRAINING 



I. INTRODUCTION 



Background 

The Military Airlift Command (MAC) 1s 1n the process of retrofitting Its C-141 and C-5A 
aircraft with a Fuel Savings Advisory System (FSA5) designed to maximize fuel efficiency. As 
aircraft are modified* all operational aircrews must be trained to operate the new fuel 
management equipment. Pilots* navigators, and flight engineers must be able to perform FSAS 
tasks as part of mission requirements. Moreover, aircrew members must be able to operate the 
system with some proficiency before attempting to operate 1t during flight. Until recently* MAC 
was faced with the* problem of bringing Its aircrews to basic proficiency on FSAS without the aid 
of ground-based training equipment. The alternative was to use the aircraft Itself on the ground 
for the Initial proficiency training. Like all operational avionics systems* FSAS would be very 
expensive to repHcate/slmulate 1f actual spare Instrument panels were procured for use with 
ground-based hardware/software for training purposes* although such an option has been considered 
and 1s discussed later 1n this report. 

For this reason, another alternative was proposed: simulation of FSAS using Inexpensive 
microcomputer-based graphics and software. Such a system would be designed to bring the aircrew 
member to a level of proficiency sufficient to permit safe operation of FSAS under actual flight 
conditions. MAC recently developed such a system and the evaluation of Its effectiveness as a 
training device by AFHRL 1s the subject of the present report. 



Related Research 

Although there has been much recent Interest 1n the use of microcomputer-based devices for 
various military training applications* there are relatively few empirical experiments assessing 
training effectiveness. Numerous reports document the application of computer-assisted 
Instruction (CAI) 1n various forms for various subject matters (usually academic). However* few 
studies have Investigated applications 1n which the computer Itself 1s used both as an Instructor 
and a simulator to provide guided practice* leading to on-task competencies. Johnson* Munro* and 
Towne (1981) and Towne and Munro (1981) have reported the transfer of performance competencies 
for electronic equipment repair skills, using computer-assisted, two-dimensional simulation, to 
performance of these tasks on actual (three-dimensional) equipment. C1cch1nell1, Harmon* and 
Keller (1982) suggested that low-cost* two-dimensional simulations were appropriate 1n training 
contexts for replicating many of the physical and functional characteristics of the actual 
equipment. Massey (1985) found that touch-actuated video screen displays of equipment control 
components were equal 1n training effectiveness to the actual equipment controls for teaching 
basic equipment operations to avionics systems maintenance trainees. In the same experiment* 
Massey found the Interactive graphics displays were superior to the actual equipment for teaching 
avionics troubleshooting skills to maintenance trainees. These studies support the use of small 
computer systems for specific types of procedures training. 

There 1s some empirical evidence to support the training effectiveness of 
microcomputer-related technology 1n aircrew task simulation contexts. Pohlman and Edwards (1983) 
demonstrated the superiority of a computer-aided* two-dimensional graphics simulation over 
Illustrated textual materials 1n the transfer of cockpit procedural skills to a flight 
simulator. Brooks (1985) reported the effectiveness of a microcomputer-based, two-dimensional 
graphics system for the acquisition of aircrew procedural skills 1n radar warning receiver (RWR) 



avionics system operations. Hageman (1983) reported the effectiveness of transferring procedural 
task competencies 1n aircrew avionics systems from a two-dimensional computer graphics system to 
operation of actual aircraft avionics systems. 

Purpose of Experiment 

The present experiment was undertaken to determine the training effectiveness and potential 
cost savings of a microcomputer-based, avionics procedures trainer as applied to a pressing A1r 
Force training requirement. The purpose of the empirical evaluation was twofold: (a) to 
determine the feasibility of employing a low-cost, part-task trainer as a substitute for 
1n-a1rcraft avionics training, and (b) to provide empirical training effectiveness data as the 
basis for cost effectiveness attainable with a low-cost microcomputer device. The scope of 
training was limited to on-task competencies for basic FSAS procedures. The experiment reported 
herein assessed the transfer of training from the experimental device to operation of the 
aircraft equipment. 



II. METHOD 



Design of Copputer-Based Instruction 

The design and development of instructional software was accomplished using TEACHER (1n 
Microsoft Basic), an English-language authoring software program developed by Capt John Queern, 
HQ MAC/DOOAE 1 . TEACHER was designed specifically for subject-matter experts who lack computer 
programming skills. The basic TEACHER program was extended to accommodate Interactive graphics 
for the FSAS trainer by Capt Frank Murray, HQ MAC/DOT, and renamed G-TEACHER. It 1s compatible 
with the Honeywell EXPER language and similar 1n instructional power and flexibility to the PILOT 
authoring system. 

A systematic approach was used 1n the design of the training software. Task analysis of the 
FSAS was conducted, from which a set of training objectives and skill proficiency requirements 
were developed. Further analysis resulted in the Identification of detailed procedures for each 
phase of FSAS familiarization and basic operation. These were further organized Into a series of 
eight proficiency modules or lessons; I.e., one lesson for each of the tasks required for FSAS 
Initial training. The tasks were (a) loading present position, (b) loading way points, (c) 
making changes 1n Initial way point data, (d) loading TACAN data, (e) loading takeoff data, (f) 
verifying takeoff data, (g) performing way point changes, and (h) selecting maximum rate of climb 
and "coupling" FSAS to the autopilot. The training objective was to get the student "on-task" as 
soon as possible by engaging the student Interactively with FSAS, as simulated by the 
microcomputer-based software. Proficiency measures were specified behavioral ly, and practice 
routines were designed Into the software for each task. Lessons were developed as stand-alone, 
self -paced Instruction. 



'Capt Queern developed TEACHER while assigned to HQ MAC/DOOAE as an A1r Force Reservist. 
It Is of Interest that the developmental work was accomplished during Capt Queern 1 s off-duty 
time, and TEACHER was Informally donated to the Military Airlift Conmand. The software was 
originally written in microsoft basic for ar\y CP/M-compat1ble computer. The program contained no 
graphics Input or output capabilities nor multi-screen or multiple Input source capabilities. 
All of the latter capabilities were developed by Capt Frank Murray, who named the 
graphics-enriched version of the program G-TEACHER (pronounced Gee-teacher) for graphics version 
of TEACHER. 
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Training Development/Delivery System 

Courseware was developed on a North star Horizon microcomputer fitted with a 12-Inch monitor 
and keyboard and a Micro Angel o graphics board. The monitor with Hghtpen (later fitted with a 
touch screen) was used to display the simulated FSAS instrument panel. Tutorial text was 
displayed on a separate monitor. An FSAS subject-matter expert, using the authoring language, 
developed all of the lesson materials. 

The same system was used for both training development and training delivery. Each student 
sat at a table containing the computer equipment and monitors. Simple Instructions were provided 
on how to Initiate lessons by making minimal Inputs via the computer terminal keyboard. Text 
displayed on the monitor explained the lessons and prompted student responses. The FSAS control 
panel (slightly enlarged from actual size) was displayed on the graphics monitor, and the 
students could simulate operating It by touching the display screen while they followed the text 
Instructions (Figure 1). Student Inputs were judged for correctness by the system, and remedial 
prompting was provided as required. The students received explanations and guided practice on 
each task. Self- tests permitted the students to assess their own competency on each task before 
proceeding to the next lesson. 




Figure 1. Low-Cost Avionics Trainer. 



Software Validation. Eleven C-141 pilots participated In a tryout of the experimental 
software. During these tiyouts, close dialog was maintained between pilots and course developers 
in order to maximize the software revision process. Although no extensive changes were found 
necessary, nearly all lessons required slight modifications. 

To check the effectiveness of the computer-based training, students in the tryout group, 
following training on the experimental device, were taken to an FSAS-equipped C-141 to see if 
they could correctly execute tasks on the actual equipment. Although this performance check was 
administered Informally, it was clear that students were able to perform the tasks. 
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As part of the try out, each student answered a questionnaire about the experimental 
training./ The questionnaire was Included In the computer program at the end of the eight 
lessons. Respondents were asked to rate various aspects of the experimental training. A sunwary 
of the responses to the questionnaire 1s shown 1n Table 1, and a copy of the questionnaire format 
1s contained In Appendix A. 

Table 1. Trainer Tryout Questionnaire Sunoary (N ■ 11) 



Rating scale Mn rating 



Trainer Ease of Use for: 1-5 (hard-easy) 

Text Information 2.9 

Light Pen Use 1.0 

Keyboard Use 3.7 

Graphic Displays 3.2 

Trainer Usefulness for FSAS Task for: 1-5 (not useful - veiy useful) 

Basic Information 4.3 

FSAS/INS Functions 3.8 

INS MODE Keys and Functions 3.5 

LOAD MODE Functions 3.9 

NAY MODE Functions 4.3 

FLT MODE Functions 4.3 

PLAN MODE Functions 4.1 

RADAR Display Functions 4.1 

Amount. of Training Provided for: 1-5 (not enough - too much) 

Basic Information 3.5 

FSAS/INS Functions 3.3 

INS MODE Keys and Functions 3.3 

LOAD MODE Functions 3.3 

NAY MODE Functions 3.1 

FLT MODE Functions 3.1 

PLAN MODE Functions 3.1 

RADAR MODE Functions 3.0 

Usefulness for Training 

Other Types of Tasks: 1-5 (not useful - very t&aful) 3.8 



In general, the opinions of pilots In the tryout group Indicate the acceptability of the 
trainer, and aircraft testing confirmed training effectiveness. Students were also asked to 
comment about any problems or Ideas for lesson Improvement. Suggestions were reviewed and 
Included 1n revised versions of the software as appropriate. As can be seen from Table 1, the 
major problem that students encountered 1n the initial torsion of the trainer was use of the 
Hghtpen. Many found 1t awkward to manipulate and Inaccurate for data Inputs via the graphics 
screen. When the Hghtpen was replaced with a touchscreen, students could Input data by finger 
touch and this modification eliminated virtually all Inaccuracies 1n student responding. 
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Experimental Procedure. Twenty-six C-141 pilots participated as subjects in tfc* empirical 
validation of the FSAS trainer. They were operationally qualified, with varying flying 
experience 1n ranks from second lieutenant to captain. Hone had prior experience or training c?i 
the FSAS. 

All subjects received general information about the FSAS in a classrooa lecture before 
participating in the experimental training. The content and tiee of lectures were controlled* 
The same Instructor and lesson material were used for all trainees. 

Subjects were randomly assigned to either an experimental group (n ■ 13) to receive training 
via the experimental device, or to a control group (n ■ 13) to receive training on the flight 
deck of an FSAS-equ1pped C-141 aircraft. 

Subjects assigned to the experimental group were Instructed to follow the Instructions 
contained 1n the computer program by practicing and self-testing on each task until they were 
confident they could execute 1t correctly and unaided 1n the aircraft. Upon completing the 
computerized Instruction, students were taken to the C-141 aircraft flight deck and given a 
criterion test on each of the eight tasks using the actual FSAS equipment. Experimental group 
subjects were tested and remediated as required on each task 1n the aircraft until achievement of 
criterion proficiency. 

Subjects assigned to the control group received FSAS training 1n the aircraft only. The* 
received the same lesson content and task sequence as the computer-trained experimental group. 
During the Instructional phase, students received guided practice on each of the tasks. They 
were Instructed to Indicate when they were confident they could perform each of the tasks 
unassisted. They were then tested and remediated until they achieved criterion proficiency. 
Thus, the content and sequence of training and testing were standardized and controlled for both 
groups. 

Task Proficiency Criteria. A standard (MAC) proficiency rating scale, based on behavioral 
descriptors, was used by the Instructor 1n evaluating the performance of both groups on the 
criterion test. A scale from 0 to 4 was defined as follows: (0) Cannot coqplete the task; (1) 
Has difficulty and requires assistance; (2) Completes the task with minimal prompting; (3) 
Completes the task without assistance; and (4) Completes the task quickly and accurately. 

The minimal proficiency rating of ^3 was specified as the performance criterion for each of 
the eight FSAS tasks. Times required for the Instructional and testing portions of the training 
were recorded for all students in both groups. All subjects were tested and rated by the same 
Instructor to preclude Interrating bias. The total training time (time on task) required to 
achieve criterion proficiency by each subject on the eight tasks was used ts the dependent 
measure of training. 



Experimental Paradigm 

A Roscoe (1971, 1972) transfer effectiveness ratio (TER) was used to assess training 
effectiveness as a function of training time to criterion performance. This design provides an 
empirically based determination of the transfer of training from the experimental trainer to the 
aircraft. 
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III. RESULTS 



Table 2 displays a comparison of training tines for the experimental and control groups. The 
experimental group required an average of 116.54 minutes per subject to complete FSAS training on 
the microcomputer system. Experimental subjects required an average of 30.15 minutes to 
demonstrate criterion proficiency on the aircraft FSAS. Average total training time for the 
experimental group was 146.69 minutes. The control group received all FSAS training in the 
aircraft. The mean training time for this group was 120.24 minutes and they required an average 
of 30.76 minutes to demonstrate proficiency during the testing phase, for an average total time 
per subject of 151.00 minutes. None of the differences between the mean time comparisons between 
the two groups, as shown 1n Table 2, was significant statistically. 



Table 2. Mean Training Time {PHnutesJ to Criterion Proficiency 
as a Function of Two Training Methods for Eight Procedural Tasks 







Required 






Required 


criterion 


Total 




training 


testing 


time on 




time 


time 


task 


Experimental (N ■ 13) 


116.54 


30.15 


146.69 


Control VJ - 13) 


120.24 


30.76 


151.00 



Transfer of Training . Applying the TE£ to the obtained training data, the following outcome 
1s achieved: 




Where: 



C * control group mean time (minutes) to criterion performance using actual aircraft 
equipment. 

X * experimental group mean time (minutes) using actual aircraft equipment (after 
pre-t raining). 

PTT X * experimental group mean training time (minutes) using microcomputer-based part-task 
trainer prior to training 1n aircraft. 
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The obtained TER of 1.04 Indicates that there 1s slightly better than a one-to-one 
correspondence between the effectiveness of the training provided by the experimental trainer and 
that provided 1n the aircraft. In short, the data Indicate the trainer was fully as effective as 
the ground-based aircraft (with Instructor) for teaching familiarization and basic operational 
skills associated with FSAS. 

Proficiency Acquisition. While minimal proficiency was a rating of 3, If the subject was 
able to complete the task quickly and accurately without prompting, a rating of 4 was given for 
the task. The experimental group exceeded the control group on the ratings. The mean 
proficiency rating for the experimental group across the eight tasks was 3.50 (SO ■ .20) compared 
to a mean of 3.18 (SD * .13) for the control group. This difference Is significant at the .001 
level of statistical probability (t » 4.85). 



Cost Effectiveness 

The transfer-of-tralnlng effectiveness of the experimental system shows the replacement value 
of the experimental device for the aircraft and provides the required empirical basis for cost 
comparisons. Cost data comparisons are predicated on the assunptlon of a one-to-one replacement 
value of the trainer for the aircraft for FSAS Initial training. 

Cost comparisons are based on several assumptions. For purposes of presentation, cost 
comparison data are only summarized In this section of the report. Appendix B provides a full 
exposition of the assumptions and derivation of cost formulas. 

Aircrew Training. The first assumption 1s that cost analysis will be limited to the training 
of the current population of aircrew members 1n the active and reserve flying units plus crew 
members attached to those units. Thus, training costs are applied to a training period of less 
than 2 years. Additional savings could be achieved If training on the experimental system were 
extended beyond current aircrews. 

A total of 4,247 aircrew members are to be trained for the C-141, Including pilot/copilot, 
navigator, and flight engineer positions. The total number of C-5A aircrew members to be trained 
1s 1,212, for a grand total of 5,459 crew members to be trained on FSAS. 

Training Hours. A standard factor of 2.5 training hours per student 1s assumed for both 
methods of Instruction. This figure 1s derived from empirical findings of the present transfer- 
of-tralnlng data. The 2.5-hour figure 1s an approximated average from the total training time 
required per student for both the experimental and control groups. Aircrews 1n both the 
experimental and control conditions were trained 1n pairs, and the assumption for cost 
comparisons 1s that this practice would be used for future FSAS training. Thus, the derivation 
of the total training hours 1s the total aircrew members to be trained x 2.5 hours divided by 2 
hours. The total training hours required for C-141 aircrews 1s 5,309 and, for the C-5A, 1t 1s 
1,515. 

Alrcraft-as-Tra1ner Costs. The use of an aircraft equipped with the FSAS system would be 
required 1n Heu of no other training device. MAC classifies the use of aircraft for ground 
training purposes as lost opportunity of revenue and charges for aircraft time accordingly 
(Appendix B). Cost 1s calculated on an hourly basis with a 24-hour celling cost. Taking Into 
account the expected training time and scheduling efficiency, 1t 1s estimated that a total of no 
more than six students could be trained In an available aircraft during a 24-hour period. The 
derived training cost per student for FSAS training In the C-141 1s $1,720.17 and, for the C-5, 
1t 1s $2,815.00. 
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Instructor/Developer Costs , The cost of Instructor time and software developer time is 
derived applying a standard hourly rate of $27.11 for a rated A1r Force officer. 



Microcomputer Training Costs. Costs associated with the use of the experimental trainer used 
1n the present study are subdivided Into hardware acquisition, Initial spare systems, and 
operating and maintenance costs (electricity, hardware and software maintenance, and magnetic 
discs). The costs shown reflect the acquisition of seven systems to train both C-141 and C-5A 
aircrew members. 

Courseware Costs. Courseware costs associated with aircraft training are for development of 
training syllabus materials to be used by Instructors 1n the aircraft. Courseware costs for the 
microcomputer version of FSAS training are for the development of FSAS simulation and training 
software. Courseware costs are computed at the standard manpower rate (rated officer s $27.11 
per hour). 



Cost Comparisons 

Training Hethod 1 - Aircraft C-141 Costs 

Aircraft Use Cost $7,305,562.00 

Courseware 542.20 



Instructor Time 
C-141 Totals 



143.926.99 
$7,450,031.19 



Method 2 - Microcomputer Trainer 
Hardware $50,400.00 
Courseware 8, 675. 20 

0per/Ma1nt Costs 3,775.72 
Instructor Time 74,366.73 



C-5A Costs Aircraft Use Costs 

Aircraft Use Cost $3,411,780.00 C-141 1,468,400.25 

Courseware 542. 20 C-5 685, 773. 84 

Instructor Time 41.071.65 

C-5 Total $3,453,393.85 

Total Cost 

C-141 + C-5A Training $10,903,425.04 $2,291,391.74 

Discussion of Training Costs. The cost comparisons clearly show the training value of the 
low-cost system for FSAS Initial training. The use of a microcomputer for FSAS training 1s a 
prime example of timely and efficient technology application. The cost benefit ratio 1s 4.76 to 
1. This cost differential 1s accounted for 1n the saving of time that would otherwise be 
required to train personnel in the aircraft on the ground. The overall cost avoidance potential 
of the microcomputer version of FSAS training as applied to training of current C-141 and C-5A 
aircrews in FSAS basic procedures 1s $8,612,033.30. 

One additional explanation 1s in order about alternative ground-based training. The original 
FSAS procurement plan provided for training using table-top trainers. These were to be composed 
of trainer-specific components (cables, fan, power converter, brackets, programs, and 
documentation) and aircraft components (FSAS control display units and computer). The aircraft 
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components would simply be FSAS system spares. During the procurement process, because of 
funding difficulties, provision for these training components was cancelled. The only remaining 
alternative was to perform training in FSAS-equipped aircraft. For purposes of comparison, 
however, the estimated costs associated with training using an hypothetical trainer are Included 
1n Appendix B under Method 3. 



IY. DISCUSSION/CONCLUSIONS 

The purpose oV the present experiment was to evaluate a low-cost, microcomputer-based 
avionics simulation as a substitute for aircraft-based training. Specifically, 1t was Intended 
to establish transfer-of-tra1n1ng effectiveness as a basis for cost benefit determination. 

The results of the experiment clearly demonstrate the microcomputer-based training to be 
equivalent to training 1n the aircraft (with instructor) as measured by training time required to 
achieve proficiency on eight FSAS procedural tasks. A one-to-one replacement cost ratio 1s, 
therefore, justified in comparing the cost of the two training methods. The effectiveness of the 
experimental training 1s further supported by the obtained task proficiency ratings. The 
experimental group achieved proficiency ratings that were reliably higher across the eight tasks 
than those achieved by the control group. 

At least three factors contribute to the effectiveness of the microcomputer-based FSAS 
training: (a) capability of the microcomputer as a vehicle for both simulation and Instruction, 
(b) software development by a non-progranroer subject-matter expert, and (c) replacement of both 
the aircraft and aircrew Instructor with a low-cost, stand-alone trainer. 

1. The Microcomputer as an Avionics Simulator/Trainer. A key factor 1n the success of the 
experimental training was the use of the microcomputer to simulate the avionics system. 
Simulation of FSAS functions made the difference between permitting the student to actually 
practice tasks as would be done 1n the aircraft and merely receiving explanations about the 
tasks. With the microcomputer as teacher/simulator, the student was put "on-task" without being 
1n the actual aircraft. The essential perceptual and cognitive dimensions of the task were 
preserved 1n the simulation. Although 1t 1s true that the low-cost simulation lacked the tactile 
or touch fidelity associated with depressing actual FSAS control buttons, the training transfer 
data bear out that touch fidelity for this task 1s Inconsequential as a determinant of training 
effectiveness. This finding 1s consistent with those of a similar studly (Pohlman & Edwards, 
1983) 1n which graphics display was successfully substituted for actual cockpit symbols and 
pushbutton switches. Massey's (1985) findings are also consistent 1n this regard. A reasonable 
explanation of these findings 1s that the essential elements of such procedural tasks are 
perceptual and cognitive rather than psychomotor 1n naturt. 

Real -time simulation of the FSAS system (flying environment) was beyond the scope of present 
requirements of FSAS basic procedures training. However, real-time flight simulation of FSAS 
operations 1s potentially deliverable via a microcomputer. This 1s a possibility MAC and other 
A1r Force users might well pursue for a variety of similar procedural tasks. 

2. Software Development . A significant element 1n the development and cost effectiveness 
of the trainer was that the software was authored by a subject-matter expert who had no computer 
programing experience. This was made possible by using the G- TEACHER authoring system developed 
at HQ MAC/DOT and represented substantial cost savings over the development of software using a 
professional programmer who would have required special training 1n FSAS functions and operations 
in order to develop the instructional materials. 
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The G-TEACHER system was well suited for the development of FSAS training material. It 1s 
not as sophisticated or powerful as many courseware authoring systems currently available. In 
the present case, however, the lack of complexity was a virtue. The work of authoring 
computer-based Instruction 1s roughly analogous to the use of word processing software. A 
general rule applies: The more power (capability) the system offers to the user, the more effort 
required to learn to use the system. G-TEACHER 1s a fairly simple system to learn and use. It 
provides the subject-matter expert who has no computer programming skills with a rea<ty tool for 
developing Instructional materials. It does not automatically ensure Instructional quality* of 
course, but the nature of the training task for FSAS 1s very explicit. The basic operations of 
FSAS are straightforward behavioral ly. The training objective was clear: Teach the student to 
perform each of the eight procedural tasks 1n logical order, permit him to practice each task, 
and provide adequate feedback to confirm proficiency. The capabilities of G-TEACHER to provide 
all of these components were very satisfactory. A more costly authoring system, 1f available, 
might have been used, assuming the Investment of time by the subject-matter expert to learn Its 
use, but 1t would not necessarily have provided more effective training. G-TEACHER warrants 
wider application in contexts similar to that of the present study. 

3. Replacement of Instructor Time. Next to the cost of the training equipment Itself, the 
greatest cost difference 1s accounted for by the fact that the experimental system required no 
Instructor except for time in the aircraft to verify task proficiency. The cost of the G-TEACHER 
computer courseware, which can tutor a student, 1s much less than aircraft Instructor costs, and 
the courseware development cost can be quickly absorbed. In this sense, the microcomputer 
becomes not only an effective Instructor, but also an extremely cheap one. 

The outcomes of the present study present an exemplary application of microcomputer-aided 
aircrew training. The empirical findings show the experimental device to be at least the 
equivalent of the aircraft 1n terms of training time to criterion proficiency. Moreover, the 
experimental group actually exceeded the standard for proficiency across the eight tasks, as 
Indicated by the mean task ratings. Student autoraony of the learning experience, combined with 
free play and feedback on tasks, may 1n this case account for the superiority of the experimental 
group 1n exceeding proficiency requirements. 

The present findings underscore the Importance of the microcomputer and Its potential Impact 
on aircrew training for selected tasks. They clearly show that the appropriate matching of task 
characteristics with low-cost training systems has large-scale cost savings potential. Aircrew 
training managers and decision makers are encouraged to exploit this potential 1n their programs. 
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APPENDIX A: TRAINER TRYOUT QUESTIONNAIRE 



Rate the quality of the training provided In this program by answering each of the questions 
below as Indicated. 

1. How easy or hard was the trainer to use? Rate the ease of use of each of the following 
features by Inputting a number frost one to five as Indicated on the scale for esch 1tea. 



Ease of use 

Text Informetlon 
Light Pen 
Keyboard 
Graphic Displays 



Very Hard 

1 2 

1 2 

1 2 

1 2 



3 
3 
3 
3 



Very Easy 
4 5 
4 5 
4 5 
4 5 



2. How useful was the trainer? Rate the usefulness of the trainer to train each of the 
basic FSAS operations by selecting a number from one to five on the scale provided. 



Trainer usefulness 
for FSAS tasks 

Basic Information 

FSAS/INS Functions 

INS Mode Keys and Functions 

LOAD Mode Functions 

NAV Mode Functions 

FLT Mode Functions 

PLAN Mode Functions 

RADAR Display Functions 



Not Useful 



Very Useful 



3 
3 
3 
3 
3 
3 
3 
3 



4 
4 
4 
4 
4 
4 
4 
4 



5 
5 
5 
5 
5 
5 
5 
5 



3. How much training did you get on the trainer for FSAS operations? Rate the amount of 
training provided for each function by selecting a number from 1 to 5 on the scale provided. 



How much training 
provided for 

Basic Information 

FSAS/INS Functions 

INS Mode Keys and Functions 

LOAD Mode Functions 

NAV Mode Functions 

FLT Mode Functions 

PLAN Mode Functions 

RADAR Mode Functions 



Not Enough 



Too Much 



4 
4 
4 
4 
4 
4 
4 
4 



4. 

tasks? 



How useful do you think this kind of trainer would be to teach other similar types of 



Usefulness for training 
other tasks 



End of questionnaire - thank you. 



Not Useful Very Useful 

1 2 3 4 5 
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APPENDIX B: COST DATA DOCUMENTATION 



The data supplied in this appendix were furnished from sources within MAC as compiled by Capt 
Frank Hurray. Standard costs documented 1n the first section below are applied using a cost 
calculation formula so that equivalence of factors is maintained across the three training 
methods being compared: Method 1, Aircraft; Method 2, Special Function Trainer (Microcomputer 
System); and Method 3, Table-Top Trainer (Aircraft spares as ground trainer). The assumptions 
and rationale for applying the standard cost data are discussed under each training method. 



STANDARD COSTS 



Electricity Cost 

Electricity cost 1s estimated at $0.10 per kilowatt-hour. 
Source: 63rd Avionics Maintenance Squadron FSAS Project Officer. 



Manpower Cost 

Manpower cost - $56,391.00 per year (standard cost for rated officer) divided by 2,080 hours 
per year ■ $27,11 per hour. 

Source : AFR 173-13, 1 Feb 85. Annual rate from Table 3.5, divisor from Chapter 3. 



Mjgber of Aircrew Members to Train 

The number of crew members requiring Initial FSAS training 1s equal to the population of 
aircrew members in the active and reserve flying units plus crew members attached to those units 
for flying purposes. 

C-141 Crew members ■ 4,247 
C-5A Crew members ■ 1.212 
Total Crew members ■ 5,459 

Source : Chief, Operations Systems Management, HQ MAC/DOTM. 



Training Hours 



A standard factor of 2.5 hours per session 1s assumed for comparison of the methods of 
Instruction. Students are assumed to be trained 1n pairs, using one Instructor when an 
Instructor 1s required by the training method. Training normally requires slightly less than 2.5 
hours, but a 2.5-hour block was used as a reasonable rounded figure for scheduled training 
considerations and cost calculations. The 2.5-hour estimate 1s consistent with training time 
requirements reported 1n this stucty. 
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Total FSAS Training Hours Required, For C-141 aircrew training, 4,247 crew members times 2.S 
hours of Instruction per student divided by two students per training session « 5,309 trainer 
hours. For C-5A aircrew training, 1,212 crew members times 2.5 hours of Instruction per student 
divided by two students per training session « 1,515 trainer hours required. (Total trainer 
hours required for C-141 and C-5A « 6,824 hours.) 

.Instructor Hours. The ratio of one Instructor hour per trainer hour Is assumed (If 
Instructor Is required by the training method). 

Source : FSAS Project Officer, 14th NASD/DOT. 



COST CALCULATIONS 

Only costs which differ between the training methods are Included In the cost calculations. 
Costs that are the same for each method (scheduling, record-keeping, classroom lecture, student 
manpower) are thus omitted as factors for calculation. 

Cost Variables Used 1n Calculations 

AC » Acquisition cost 

OC » Operating and maintenance cost (hourly) 

TH - Trainer hours (5,509 plus 1,515) 

CC * Courseware cost 

IH « Instructor hours (same as TH) 

MC « Manpower cost (hourly ■ $27.11) 

PS « per student aircraft cost 

NS = Number of students 

Calcul atlons. The general formula for cost calculation Is: 

Cost «= AC = (TH x OC) « CC (IH x NC) 

The cost calculation for Training Method 1 (Aircraft) differs 1n the hardware cost. Cost of 
training 1n the aircraft 1s computed on a per-student basis, rather than an hourly basis, 
resulting 1n the following equations: 

Cost (C-141) « (NS x PS) + CC + (IH x MC) 

Cost (C-5A) » (NS x PS) + CC + (IH x MC) 



TRAINING METHOD 1 (AIRCRAFT) 



Hardware Cost 

Acquisition, operation, and maintenance costs of training 1n FSAS-equ1pped aircraft are not 
relevant to training cost considerations since those costs are not attributable to training. 

However, an understanding of MAC policy on use of aircraft for ground-based training 1s 
essential as a basis for cost comparisons. MAC's operating budget comes from the Airlift Service 
Industrial Fund (ASIF). To replenish the ASIF, MAC uses Its aircraft to generate revenue by 
"renting" or otherwise charging for aircraft use by 000 and other agencies. When aircraft are 
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not available for atrHft services, users must contract for commercial services at higher rates, 
and MAC must obtain additional Congressional appropriations to continue Its operations. Thus, 
non-revenue-produdng usage of aircraft, even for MAC's own activities, 1s accounted for as lost 
opportunity for revenue. A cost formula, specifically derived for non-revenue use of aircraft, 1s 
applied. The use of aircraft for ground-based training 1s considered non-revenue-produdng 
activity and 1s charged accordingly. This cost 1s calculated on on hourly basis with a 
"not-to-exceed" celling for a 24-hour period. For example, for the C-141, the hourly charge 1s 
$3,125.00, not to exceed $10,321.00 per 24-hour period. Obviously, considering the number of 
students to be trained on the FSAS, the per-day rather than per-hour rate 1s more advantageous. 
The nt»ber of students trained per day (allowing for estimated training time and reasonable 
scheduling efficiency) 1s divided Into the celling cost of the aircraft per day to derive the 
hardware cost per student. Note that the cost for training 1n an aircraft varies with the type of 
aircraft. 

Hardware Cost - C-141: $3,125.00 per hour, not to exceed $10,321.00 per 24-hour period. 

C-5A: $7,770.00 per hour, not to exceed $16,890.00 per 24-hour period. 
Source ; Chief, Cost and Economic Analysis Division, HQ MAC. 

Accounting for the normal availability of aircraft and crew members for training, an average 
of six students per training day per aircraft can be expected. The following per-student aircraft 
cost results: 

Per-student hardware cost - $10,321 divided by 6 - $1,720.17 per C-141 student; $16,890 

divided by 6 ■ $2,815.00 per C-5A student. 

Source: MAC FSAS Training Project Officer, 14th MAS/DOT. 

Courseware Cost. Courseware development cost would Involve the creation of a lesson plan to 
be used by the Instructor while training the student using the table-top trainer. The lesson plan 
for this training option would be the same as that used for training using FSAS spare equipment in 
a ground-based, part-task trainer. 

Courseware cost - 20 Instructor hours time $27.11 (rated officer hourly rate) » $542.20. 
Instructor Cost 

The Instructor cost for aircraft training 1s computed on an hourly basis at instructor hour 
per trainer hour. The hourly rate 1s based on the standard cost of a rated officer ($27.11 per 
hour). 

Cost Calculation 

Tlie training cost calculation for the aircraft method of training 1s as follows: 
Cost (C-141) « (NS X PS) + CC + (IH X MC) 

Cost (C-141) - (4,247 X $1,720.17) + $542.20 + (5,309 x $27.11) 
Cost (C-141) - $7,450,031.19 



Cost (C-5A) « (HS X PS) + CC + i%H X MC) 

Cost (C-5A) « (1,212 X $2,815.CSJ + $542.20 + (1,515 x $27.11) 

Cost (C-5A) « $3,453,393.85 

Total Cost Both Aircraft « $10,903,425.04 
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METHOD 2 (MICROCOMPUTER SPECIAL FUNCTION TRAINER) 



Hardware Cost 

Acquisition Cost, Seven sets of hardware were purchased to support the FSAS training 
program: one each for four C-141 Kings, one for the C-5A W1ng, one each for two Mixed C-141/C-5A 
Wings. One set of hardware, owned by HQ MAC prior to the development of FSAS training, was 
designated a development and spare trainer to support the effort. The microcomputer-based special 
function trainer 1s a programmable device and not Intended to be used only for FSAS training. 
Upon completion 0 f FSAS training, the trainers will continue to be used for aircrew training. 
Therefore, the acquisition cost of hardware should be amortized over several training projects. 
The nunber and duration of future training programs are unknown at present, and the cost of the 
hardware 1s amortized only over the present FSAS project. Thus, the cost of equipment acquisition 
1s overstated for this training method: 

Cost per set of trainer hardware: $ 7,200.00 

Cost for seven sets of training hardware: $50,400.00 

Source ; HQ MAC/DOT message. On file at 63MAW/D0T. 



Operating and Maintenance Cost 

The operating cost of the trainers 1s based on the cost of electricity used to power 
equipment, the cost of floppy discs replacement, and the maintenance requirements for the 
equipment, based on experience accrued with the first system placed 1n operation. The exact power 
consunptlon 1s not available but 1s estimated to be 443 watts based on the maximum current ratings 
of all system components. Floppy discs require replacement typically after 200 hours of use on 
the special function trainer. 

Electricity - .443 kilowatt x $.10 per kilowatt hour « $0.0443 per trainer hour* 

Floppy disc cost - $1.80 pes* disc divided by 200 hours per disc « $0,009 per trainer hour. 

Maintenance cost - $500.00 per 1,000 hours of operation « $.50 per trainer hour. 

Operation/Maintenance Cost - $0.5533 per trainer hour. 



Courseware Costs 

A rated officer, Inexperienced 1n computer programming, produced the courseware for the 
special function trainer. Part of his time was spent 1n learning to use G-TEACHER, the authoring 
language used for courseware development. This represents a one-time cost for each courseware 
author. This one-time cost has been Included which overstates the cost of this training method 
for the FSAS application. Phases of development are as shown: 

Familiarization with Software: 40 hours x $27.11 « $1,084.40 
Production of FSAS lessons: 240 hours x $27.11 « 6,506.40 
Revision of lessons: 40 hours x $27.11 « 1,084.40 

Total Cost of Courseware $8,675.20 
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Source : MAC FSAS courseware author, 14MAS/D0T. 



Instructor Cost 

FSAS training via the microcomputer-based special function trainer precluded the requirement 
for a live Instructor, except for the time 1n the aircraft used to verify proficiency on the 
actual equipment. The average time per student required to demonstrate proficiency 1n the 
experiment was 30.15 minutes. The calculation of Instructor cost, therefore, 1s: 30.15 minutes 
(.5025 hour) x MC ($27.11) x NS (5,459) » $74,366.73 



Aircraft Use Cost 



The cost of proficiency verification testing 1n the aircraft 1s calculated based upon the 
per-student cost of aircraft training. The students trained 1n the aircraft required 2.5 hours 
of training. In the C-141, the per-student cost was $1,720.17, or $688. 07 per hour. The cost 
derivation for the testing of Method 2 trainees 1s: 30.15 minutes (.5025 hour) x $688.07 * 
$345.75 per student. Using the same derivation for the C-5A rate, the cost 1s $565.82 per 
student. The cost of testing applied across current aircrews fs: 

C-141 costs: 4,247 students X $345.75 * $1,468,400.25 
C-5A costs: 1 ,212 students x $565.82 « 685,773.84 
Total aircraft use cost for testing: = $2,154,174.09 



Cost of Training (Method 2) 

Hardware $50,400.00 

Courseware 8,675.20 

Operating/Maintenance 3, 775. 72 

Instructor 74, 366. 73 

Aircraft Use Cost 2,154,174.09 

Total $2,291,391.74 



METHOD 3 (TABLE-TOP TRAINER) 

Cost estimates for this method of training are documented as shown. However, the training 
method Itself 1s hypothetical since table-top trainers are not available. Estimates are 
presented here for cost comparison Information only. Under provisions of the original FSAS 
procurement plan, extra FSAS components were to have been provided as spares for ground 
training. However, when funding was reduced, this provision of the procurement was cancelled, 
precluding this training option. 



Hardware Cost 

Acquisition Cost . The table- top trainer would consist of two types of components: 
trainer-specific (cables, fan, power converter, brackets, programs, and documentation) and 
aircraft components (FSAS control /display unit and computer). The trainer-specific components 
would not be suitable for other uses following FSAS training and would be discarded. The cost of 
the trainer-specific components, therefore, would be directly attributable to training. The 
aircraft components would be purchased in addition to those otherwise required. After training, 
however, aircraft components could be used as additional spares. Since the additional spares 



would not be part of the Identified logistics requirement, their cost Is attributable entirely to 
training. The cost of trainers 1s calculated based on the same number of special function 
trainers shown In Method 2 (seven trainers). 

Trainer-Specific Components - $36,190 x 7 » $253,330.00 

Aircraft-Specific Components - $44,700 x 7 « $312,900.00 

Total Hardware Acquisition Cost $566,230.00 

Source ; C-141 FSAS Program Manager, WR-ALC/MMSH 

Operating and Maintenance Cost 

Hardware operating cost Is limited to electricity used to power the components during the 
training. An exact power consumption rate Is not available but Is estimated to be 2,220 watts. 
Hardware maintenance cost Is not available but could be expected to be significant since all 
components would be of new design. Components of this type typically exhibit high failure rates 
early 1n their life cycles until the design *mafrures." 

Operating Cost: 2.220 kilowatts x f 0 per kilowatt hour * $0,222 per trainer hour. 

Maintenance Cost: Unknown. Assuirtf ^ be zero which understates the cost of this method. 

Source: FSAS Systems Manager, 63rd Avi^nl, Maintenance Squadron. (FSAS table-top trainer 
current used estimated to be 17 to 20 amps). 

Courseware Cost 

Courseware development cost would Involve the creation of a lesson plan and would be 
equivalent to that of Method 1 (Aircraft Training). 

Courseware Cost: 20 Instructor hours x $27.11 per hour * $542.20 

Source: MAC FSAS Training Project Officer, 63MAS/D0T. 

Instructor Cost 

Instructor cost for this method 1s computed on the same hourly basis as for Method 1; I.e., 
$27.11 per hour. 

Cost Calculation 

(Costs are totaled for both C-141 and C-5A FSAS Training). 

Cost (Table-top training) * AC + (TH x OC) + CC + (IH x MC) 
Cost ■ $566,230.00 + (6,824 x $.222) + $542.20 + 6,824 X $27.11) 
Cost (Table-top training) - $753,285.77 



SUffli&Y OF COSTS 

The total cost of FSAS training via each of the methods elaborated in this appendix 1s shown 
below. The cost Includes FSAS training for both C-141 and C-5A aircrews. 

Method 1 (Aircraft) $10,903,425.04 

Method 2 (Microcomputer-based trainer) $ 2,291,391.74 

Method 3 (Table-top trainer) $ 753,285.77 
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